Shannon’s Entropy of The Stochastic Context-Free Grammar and an Application to RNA Secondary Structure Modeling

Stochastic context-free grammars (SCFG) have been used in RNA Secondary structure modeling. An SCFG consists of a set of grammar rules with probability for each. Given a grammar design, finding the best set of probabilities that yield optimum performance can be challenging. Although current Expectation Maximization (EM) MaximumLikelihood (ML)-based model training approaches have been effective, there is no guarantee that they provide parameter sets for the grammar to have optimum performance. In this work, An analytical measure of the SCFG space, denoted here as Grammar Space (GS) entropy, is introduced and calculated for various SCFG models in the literature. It is shown that more accurate models have lower GS entropy. Finally, based on the GS entropy, a novel RNA structure model training method is proposed.